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ABSTRACT \ 

\ A guida to tha iaplaaantation of tha Entry Laval 
Skills Prograa (ELSP) and a conceptual framework for evaluation 
rasaarch is prasantad. Attention is directed to strategies for the 
attainment of goals and management of the ELSP project, which is a 
developmental prograa for frashaan students who have not acquired the 
full range or level of cognitive skills needed for college. Overall, 
the project was designed to provide professionals, and particularly 
women and minoritiai at the postdoctoral and predoctoral levels, the 
opportunity to design and iaplaaent the evaluation research coaponent 
of tha ELSP. The competency-based skills prograa it based on the 
planning, aanageaent, and evaluation (PME) systeas concept to 
instruction and student testing using .a standardisad pre- and 
post-test battery and a series of criterion-referenced tests (e.g., 
the Cooperative Guidance and Placeaent Prograa battery). The role 
that a aanageaent information eyetem (MIS) plays in ELSP 
implementation and developaent is addressed, and a MIS open-systeas 
modal that can be utilised in the curriculua developaent process is 
presented, along with a theoretical modal for the development of a 
PME plan for the ELSP. The process for the flow of information and 
data aanageaent is also described. The major functions of the PME are 
identified and defined: planning, organising, coordinating, 
monitoring, end assessing. Additionel objectives of the prograa— to 
assist participating institutions in the developaent of instructional 
systeas (i.e., instructionel stretegies, recordkeeping, end classrooa 
aanageaent)— are also noted. Various questionnaires, foras, flow 
charts, and data elements are appended, along with a bibliography. 
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% INTRODUCTION 

The Entry Level Skills Program Is a cognltlvely oriented developmental 
education program In communications skills, mathematics and science developed 
In collaboration with approximately 23 developing Institutions. 

Generally, the ELSP reflects the philosophy and educational practices 
of competency-based education and a number of other current approaches to 
teaching/learning, Including mastery learning, Interdiscipl tnartty, 
programmed instruction, and personalized Instruction. The unique features 
of the program Include: the preparation of disadvantaged students to cope 
with standardized tests as a normal part of the Instructional process and 
related support s 'stems; focus on the mastery of learning at a level in- 
dicative of moving disadvantaged students Into mainstream higher education 
and, ultimately, mainstream Jobs. Moreover, ELSP, emphasizes a holistic 
approach to curriculum design and evaluation. 

Presented here Is a guide to the Installation of the ELSP, and 
an Inst I tut tonal -wide systems approach that will guide the project through 
various levels of Implementation. 

The ELSP Implementation Guide includes an overview of the ELSP and a 
conceptual framework for Evaluation Research. The guide focuses on managing 
the ELSP project and emphasizes strategies for the attainment of goals. 

The Institute for Services to Education, Inc. (ISE) is well qualified 
to provide the services stipulated in this guide. Since its establishment 
/ In 1965, the f Inn has provided technical Assistance to more^than one hundred 

I college* and universities ranging In size from 400 to 40,000 students in 

\ 

thirty-six states and the aggregate territories of the United States. 
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The ISE staff Includes Individuals with diverse training and experiences 
In the developing Institutions as well as other major Institutions. 

ISE does not separate faculty and curriculum development » but rather 
combines these two activities as they relate to the ELSP. The faculty Is 
developed by Increasing Its capabilities in'curriculum development using 
a team approach. Faculty representatives are Involved In the following: 
development workshops; establishment of course goals and objectives; and 
In the writing of course modules and accompanying criterion-referenced 
tests. 
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The Entry Level Skills Program 

Overview ^ 

The Entry Level Ski lis Program is a response to the need to translate 
the promise of equal opportunity and equity for the disadvantaged student in 
postsecondary education into a programmatic approach that guarantees results. 
Ultimately the program aims to: 

* Improve student performance scores on standardized tests, 
Including increasing students 1 median performance in 
reading, and Improvement in basic computational skills. 

• improve written auditory and verbal communications ski 1 Is 
of entry level students. 

• reinforce the abilities of the students to achieve and 
persist In a college environment through structured 
learning of academic procedures and processes. 

* improve teaching strategies and the use of alternative 
approaches to Instruction by assisting teachers in re- 
writing or supplementing their course outlines-syllabi to 
reflect standardized test content skills. 

' • assist institutions in implementing procedures for carrying 
out ski ll-bullding efforts within the framework of their 
administrative syslaru-J^oJudfTig "fTxfng of responsibility 

_.at evenr+evel involved, the setting up (In collaboration with 

researchers and evaluators) of manageable recordkeeping, 
monitoring, and reporting activities and the documentation of 
system changes as required. 

Implicit in the above set of objectives are a number of assumptions 

which are fundamental to making the program work. The ELSP recognizes the 

utility of standardized tests for distinguishing the better student from 
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the poorer student and, thereby, serving as a source of guidance for 
instructional planning. Of equal Importance, the program underscores the 
Interdlsclplinarity of the concepts, processes and skills assessed through 
• the use of such tests. That Is, scholastic aptitude tests, whether in 
reading, mathematics or science, tend to examine a common set of verbal, 
analytical and critical thinking competencies, as well as specific com- 
petencies in test-taking. Further, mastery of such competencies requires 
• n obiectlves-driven le arning system which facilitates systematic diagnosis 
of where s tudents are and the use of the results of diagnosis to prescribe 
the prope r match between student needs and instructional plans. 

Hence, the heart of the Entry Level Skills Program In concept and at 
Its^rrent level of development is systematic student assessment within 

the framework of an instructional management and reporting system ultimate! 

* 

aimed at guaranteeing equality of output for the disadvantaged In higher- 
education. 
Expected Outcomes 

The primary expected outcome of the ELSP Is an improved and expanded 
learning system which consists of a teacheris manual, syllabus, student 
manual for each of the discipline artas, a PME system, an appropriate 
test package and a monitoring system. Related outcomes of critical im- 

» 

portance are: 

0 establishment of qualitative data bank, which includes detailed 
faculty logs, progress reports prepared by the ELSP director, 
the research coordinator, the ISE research' staff and the Faculty 
and Curriculum Development Program staff; minutes of thel local 
task force; reports of the ISE task force and an external panel 
of experts; 

quantitative data base addressed to the following: achievement 
as measured by the standardized test battery, criterion- 
referenced tests, and GPA's: attendance, attrition, and demo- 
g raph i c/psychosoc i a 1 charac te r i s 1 1 cs ; 

an improved student information and instructional manager* nt 
system in each of the participating institutions; and 

improved faculty and student performance in each participating 
institution. 

O ,2 
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Evaluation Research: A Conceptual Framework 



In light of the current stress on total Instructional accountability and 
• documentation of program Impact, It appears that the evaluat'on research compo- 
nent of the entry level skills program must meet two Important requirements: 

• Facilitate ongoing assessment of student needs, and (2) ongoing decision- 
making regarding performance objectives, course content, Instructional 
strategies and placement of students. 

• • • 

• Facilitate systenatlc determination of the Impact of the curriculum 
model on students enrol led in the program. 

Implicit In these two requirements Is a set of evaluation principles, or in 

essence, a philosophy of evaluation research, which deserves mention at this 

point. 

• ^valuation research should be based on a sound understanding of (1) 
the purpose of evaluation; <2) what Is to be evaluated (i.e., inputs, 
processes, or outcomes); and (3) hew the results generated by evaluation 
research are to be used. 

• Evaluation research should be designed to facilitate (l) Instructional 
decision-making and (2) the determination of the effectiveness of Instruc 
tlonal programs. 

• Evaluation research should be, multivariate In nature and comprehensive 
Decisions about the readiness of students to proceed to various points 
In the Instructional program should be based on a variety of Indicators; 
for example, GPA's, standardized test scores and behavioral ratings. 
Hence, the evaluation research approach should exemplify a sound synthesl 
of the use of both quantitative and qualitative data . 

• Evaluation research should be systematic and vigorous , beginning with the 
Initial point of Instructional planning and continuing through program 
development, implementation and monitoring. 

• Evaluation research should be dynamic , not static. As the Instructional 
program unfolds, unanticipated events in the teaching- learning process 
may call for changes In the original plan. 

• Evaluation research should be growth-oriented . The evaluation process 
should force us to think through what we're doing, to ask new kinds of 
questions and, perhaps, to propose new kinds of answers. 

(Chart 1 summarizes a number of basic concepts which facilitate the establishment 



of a viable evaluation plan.) 
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CONTEXT: 








^ Professional^ 





INPUTS 


PROCESS-FORMATIVE 

• 


OUTCOMES-PRODUCTS 


Tine 
Money . 
Instruction 
Leadership 
Goals 

Communication 
Learners 
Resources 
Curriculum 
Services 
Procedures 
Environment 
. Community 


Schedules, Flexibility, Utilization 
Budget Allocation, Expenditures 
Dlfferentlatlon-Utlllzatlon-Organlzatlon 
Roles, Decision making. Power groupings 
Clarlty-Utlllzatlon-Relatlonshlp % 
Fl ow-Access-Mechanl sms 
Organl za 1 1 on-Partl c 1 pa tl on 
Utillzatlon-Adequacy-Relatlonship to goals 
Organl za tlon , Fl exl bl 1 1 ty-Rel evance 
Adequacy-Acceptabl 1 1 ty-Operatlon 
Access-Acceptabl 1 1 ty-Operatlon 
Clar1f1cat1on-Pol1cy-Access 
Organl za tl on-Ut 1 1 1 za tl on- Fl ow 


Efficiency-Balance 

Utilization-Adequacy 

Pupil gain-Performance 

Amount-Nature-Change 

Functional Impact, Morale 

Unders tandl ng- Support 

Gain, personal /Substantive 

Utilization- Impact 

Learner gain, renewal, quality 

Amount-Impact-Balance 

Responsiveness-Action-Use 

Utilization-Adaptation 

Involvement-Support 



CONSTRAINTS: Parameters 
_ Value differences ^_ 
Priority ambiguities 
Power- Influences 



A second characteristic implicit In an evaluation research plan Is the systens 
approach. This approach becones the basic theory that guides the development 
of the evaluation plan. 

Needs analysis, feedback, the establishment of performance tasks, separating 
out component tasks are all characteristics of the systems modality. The systems 
modality also raises three basic questions essential to rational planning for 
evaluation research: 

* Where are we going? 

* How are we going to get there? 

* How will we know when we have gotten there? 

Based on the above requirement, the ELSP evaluation research model at the local 
level should be geared towerd the utilization model (Patton, 1980). Emphasis 
should be on the evaluation of effort, process «nd treatment. , 

The effort approach Is designed to provide an assessment of all entitles 
required to successfully accomplish the goals. This Includes— time, funds, 
space, equipment, etc., all factors that are necessary to accomplish the task. 
The process approach centers around the basic question of what- Is the present 
procedure. It Is basically the creative component describing, developing, 
defining, and re'infng current Implementation and practice, both quantitatively 
and qualitatively. Finally, the treetment approach measures intervention, 
the variable that Is to make the difference, to what degree (if any), for whom 
and under whet circumstances. 

This conceptual framework represents various conceptualizations of the 
evaluation process (e.g., Stuff lebeem,. Krathwohl, Taba, Campbell, and Patton). 
Although they differ in detail, each demonstrates* systems approach to evalua- 
tion and the need for the evaloatlon process to measure more than output. 
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In educational research, outcome evaluation Is represented by pretest 
versus posttest performance on standardized achievement tests, this kind of 
outcome Information Is extremely Inadequate. Simply knowing that outcomes 
are high, low, or different, does not tell decision-makers very much about 
what to do. What is missing is Information about the actual Implementation 
of the program being evaluated. Thus, where outcomes are evaluated without 
knowledge of Implementation, the results seldom provide direction for action 
because the decision-makers lack information about what produced the out- 
comes. 




MANAGING THE ELSP PROJECT 



Institute for Services to Education's primary focus In the installation 
of the Entry Level Skills Program (ELSP) Is to provide technical assistance 
In the development of a Management Information System (HIS) that will guide 
the project through the various levels of Implementation. 

Prior to program Implementation, !SE will conduct an assessment of. 

a 

the Institution's MIS capability and make recommendations \ hat are com- 
patible with the existing MIS resources. 

A brief explanation of the pivotal role that a sound MIS plays In 
ELSP Implementation and development follows. An MIS open-systems model, 
that can be utilized In the curriculum development process, Is presented 
along with a theoretical model for the development of an ELSP Planning 
Management and Evaluation (PME) plan. The process for the flow of In- 
formation and data management is also described. Annent'Ix C) 
The Role of MIS In ELSP Implementation 

A Management Information System (MIS) Is a collection of facts, 

/ 

procedures, people and (often) machines which develop information 
designed to serve as a basis for decision making and policy formation. ' 
It Is important to realize that a MIS cannot work effectively without 
the involvefl^ent and commitment of the various people that play an 
essential role in the institution's decision making process. These 
Include: students, faculty, counselors, data coordinators, program 

r 

directors, ISE consultants* the academic dean, and the president of 
the Institution. Involvement and commitment, In this context, can be 
defined as a thorough understanding of the ELSP objectives as well as 
an understanding of the importance of generating and maintaining ac- 



-7- 14 



curat* data that will facilitate the attainment of the program's 
Instructional and administrative goals. 

Another basic requirement to the proper functioning of a HIS Is 
the accuracy and timeliness of Information. This Is quite an Important 
aspect as It Is^ not uncommon for a counterproductive situation to exist 
In a program for some time before It Is brought to the attention of 
program managers. It fs also not uncommon for the situation to con- 
tinue for a period of time before corrective action Is -Implemented. * 
Herein Hes-One of the basic values of a HIS. Situations that require 
corrective managment action are detectable sooner and they can be 
clearly and accurately defined. 

> 

Detection, of course, Is only one side of the coin. Detection of 
problems and/or opportunities demand decisions. Evaluation of the 
Implications and results of decisions are critical areas of the HIS. 
Uncertainties underlie every decision. The bigger the decision, the 
more desirable It becomes to predict and offset risks. As a management 
decision begins to Impact on the current operations of the ELSP project 
an effective HIS should be able to provide Immediate feedback to program 
managers concerning the results of their actions. As a result, ad- 
justments or corrections to refocus on project objectives can be made 
on a more timely basis than is ordinarily possible. 

It cannot be stressed strongly enough that a HIS cannot be Im- 
plemented until users are properly trained. Proper training ensures 
conformity with project guidelines and also fosters an ambiance of 
security .no knowledge which Is prerequisite to Involvement and commitment. 
A necessary focus of ISE is therefore to provide training and on-going 
assessment of the management and instructional aspects of the ELSP. 

• . ' . I 
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As with any other successful undertaking, success lies In the finding 
of e proper balance between responsibility and authority. This brings the 
program Into reality. A vital first step Is to organize personnel and 
policies. Decision makers and information users must be informed and 

thoroughly convinced of the need for the system. This, In Itself, 

I 

makes Initiation of a MIS design high In both priority and Importance. 

The following description of an open-systems model, is. presented 
as an example of a functional MIS. 
A Management Information System Model 

Figure 1 illustrates a simplified picture of an open system model 
containing the basic components of a management information system (MIS). 

The HIS Is made up of five systematic Information loops which feed 
information to decision makers. They are: a "market future" loop; an 
"external feedback" loop; an Internal "quality control" feedback loop; 
an internal "personnel support* feddback loop, and a "pending resources" 
loop. Based on systematic information received from the five loops, 
the decision makers can carry out an on-going process of institutional 
development that Is in balance with the environment. 
\ The special characteristics of the information lo^^and the 
decision makers' control mechanism which make up the HIS are as follows: 

\ The "Market Future" Loop . This Information cycle closely monitors 
the Changing needs of the .market places (which receive graduates) by 
accumulating such information as specific person* resource projections, 
new work technologies, changing college admission requirements and job 
skill breakdowns* 

The "External Feedback" Lo op. This information cycle informs decision 
makers about the level of success the output or 'student) has on the market 

-3- 16 



FIGURE I 
MIS OPEN SYSTEMS MODEL 



EXTERNAL FEEDBACK 




place. This type of feedback can be obtained through follow-up studies, 
telephone calls and letters exchanged between educators and employers, 
discussions with graduates and on-site visits by educators to those 
institutions which receive graduates. 

The Internal Feedback "Quality Control :l Loop, This information cycle 
provides data on the effectiveness of the teaching- learning process as 
derived from comparing test scores with pre-determi ned objectives. The 
test scores can be from standardized tests, criterion-referenced tests or 
teacher-made tests. Measures of the teaching- learning .process are im- 
portant because they inform the teachers and administrators about student 
achievement, and classroom instruction. However, the measures do not 
necessarily tell the educators if they are teaching the right thrngs in 
the first place. The "market future 11 loop should be depended upon to 
provide information about program content. 

The Internal Feedback "Personnel Support" Loop. In order for a pro- 
gram to be successful, there is a need for support from the people who are 
directly involved, such as parents, students, teachers and administrators. 
The "personnel support" information cycle provides data on the degree of 
backing or resistance that key individuals lend to specific programs. 
Programs that otherwise seem desirable and feasible as determined by 
information loops may fail completely if sufficient backing is not 
received from personnel. In fact, a program is doomed to failure if full 
support is not provided by decision makers. 

The "Pending Resources" Loop. This information cycle provides decision 
makers with a picture of the human resources and material resources that a 
school will receive over a period of time. These data set parameters around 
what Is economically feasible in the way of projects for the students 
entering the education system. 

-il- 
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The decision makers of the school serve as the "control mechanism" 
for the MIS and are recipients of data derived from the five systematic 
Information cycles already defined: Information on the changing nature 
of the market place; Information comparing Internal measures of student 
performance against pre-determlned objectives; Information on what happens 
to the output once it arrives on the market place; Information on the 
availability of resources to support specific programs; and information 

• • • 

on the levels of support and resistance found among the personnel who will 
be directly Involved or affected by the program. 

PLANNING, MANAGEMENT AND EVALUATION (PME) : A THEORETICAL MODEL: 

The planning, management and evaluation system (PME) that can be' 
utilized for the ELSP, can be described as a process of decision making 
leading to the Implementation and accomplishment of project goals by the 
execution of a set of tasks by people within time, cost and performance 
specifications. The major functions of the PME can be divided into five 
categories and defined as follows: 

Planning: a set of initial definitions about the allocation of 
resources and human effort to an optimal set of tasks, to attain 
specific desired goals. 

Organizing; Involves the arranging of selected people In patterns 
of relationships relative to authority, responsioi 1 i ty, roles and 
accountability to facilitate accomplishment of desired goals. 
Coordinating: involves the employment of diverse forms of human 
Interaction aimed at leading, motivating and guiding people in 
performance of their tasks. 

Monitoring: involves the detection of what Is actually happening, 

-12- 
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the analysis of the deviation from what is actually occurring 
and Its resulting solution (decision) > and the implementation 
of corrective actions to ensure successful accompl ishement of the 
goals* 

Assessing: involves the development of procedures and necessary 
Instrumentation for gathering data about the performance of the 
development/management systems for the purpose of .decision-making 
about Improvement. 

Some specific examples of actions that should be carried out within 
each of these functions by either faculty, administrators or both, follows: 

The planning function Is the first and most essential function, and 
the quality of the system depends on the thoroughness with which it is 
performed. The key components of the planning function are: 

* Setting goals 

* Specif/Ing work to be done 

* Descriptive flow of the tasks 
0 Determining time schedule 

* Determining costs and people- resource needs 

* Developing a budget 

The organizing function of management Is concerned primarily with 
assignment of job responsibility. The components of this function are: 

* Definition of Jobs 

9 Assignment of specific tasks 

* Establishing lines of responsibility and accountability 

* Establishing working relationships 

The coordinating function of management is essential to the completion 
of the system objectives and primarily involves the human interaction of 
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the managment system. It includes? 

c 

9 Developing methods of communication 

9 Identifying needs of the staff or other people who play a role 
fn the Instructional process. 

9 Developing a reward system based on the needs assessment. 

Of these, communication Is the most crucial element. It must be an 
open two-way process. Managers must have specific methods of .getting feed- 
back and should also have a mechanism for providing relevant Information. 

Monitor Ing Is the next function of management. Monitoring is pri- 
marily concerned with Identifying any deviation from the plans of the 
system. Monitoring Includes: 

9 Identifying the deviation between what Is planned and what is 
happening, through the communication process 

9 Clarifying problem areas 

9 Analyzing problem areas 

9 Collecting the Information 

9 Drawing conclusions from the Information 

9 Making decisions about the system 

Assessing is the last function and is concerned primarily with 
the design and process for gathering data about the development/management 
system for the purpose of deciding what part of the system should be 
Improved. Assessing includes: 

9 Defining the purpose of assessment 

9 Determining the kinds of information that should be collected 

9 Determining how the information should be collected and analyzed 

9 Collecting the information 

9 Drawing conclusions from the Information 

9 Making decisions about the system 



Once all five functions (planning, organizing, coordinating, monitoring 
and assessment) have been applied, managers — be they faculty, administrators, 
curriculum developers or other staff — will have a detailed operational 
plan, be organized to comlete the project, have a coordinating system 
established, be Informed as to progress and have a specific design for 
assessing the management of the entire project. 

ISE management consultants will focus special attention on assist- 
ing participating Institutions In designing an efficient PHE system that 
accomplishes the objectives of the Entry Level Skills program. Figure 2 
depicts the Concurrent phases of a PHE that will be tailor-made for each 
participating institution. Chart 2 Illustrates the functions of the PHE 
system that can be used as an Initial discussion Item In developing the 
ELSP Institutional PME. This chart also Itemizes information pertinent to 
each function within the ELSP project. 
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FIGURE 2 
ELSP IMPLEMENTATION MODEL 
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DATA MANAGEMENT: 

The collection, organization, storage and retrieval of data In useable 
form Is critical to the successful Implamentat Ion of the CLSP project. 
Consequently, the data management component of the project will Include, In 
the Initial stage of development, an Information systmn designed to generate 
Instructional management reports, administrative reports, and flowcharts 

summarizing the documents, personnel and functions of the data management 

• • • 

system. Additionally, to facilitate the manual and computerized data 
management effort* a data elements directory and coding system will be 
developed. 

ISB'^sultajMeAttll assist In establishing procedures for the 
manege»eH< v of/fiata and will provide necessary forms and train appropriate 
staff In the collection and synthesis of project Information. 

Following is a brief description of the Instruments utilized to 
collect program data. * 
instrumentation: 

Reflective of the goals and objectives and underlying assumptions 
of the overall curriculum model, the. assessment program of the ELSP k 
Includes both the use of standardized and cr Iter Ion- referenced tests. t 

The pre-test package includes the Comparative Guidance ar.d rUw*iient 
Test Battery, the ISE Student Information Survey From and the ISE Faculty # 
and Administrator Survey Form. 



25 

1/ 



erJc 



CHART 2 

PLANNING. MANAGEMENT. AMD EVALUATION 



COMPONENT 


data COLLECTION 

INSTRUMENT 


PERSONNEL 


1. Planning 

Resources 
Hunan Efforts 
, Task 

Tin* Schedule 
Budget tug 


General School 
Characteristics 
Questionnaire 
Component 

Di rector • sNn terv tew 
Schedule 


Decision-makers 


II. Assessment 
Pur post 

Determine data to 

be collected 
Instrumentation 
Data Analysis 
Decision-making 


Needs Assessment 

Questionnaire 
Interview Guide 


Program Director 

Decision-makers 

Data Coordinator 
Instructors 


III. wOOrdlnatino 

Hat hods of Communi- 
cation 

Staff Needs 

Program Instruction 
Process 

Reward System 


Organizational Chart 
Needs Assessment 


Decision-makers 


IV. Monitoring 

Identify Problems 
Clarifying Problem 
Area 

Analyzing Problem 

Area 
Developing and 

implementing 

corrective 

measures 


Instructor's Charts M1-M6 
Criterion Reference Test 
Faculty Log 


Data toordlnator 
> ns t rue tors 


V. Organizing 
uerine Jobs 
Assign Specific Tasks 
Lir^e of Responsibility 

a^d Accountability 
Work Relationships 


Reeds Assessment 


Decision-makers 



( 



CHART 2 (continued) 



CLASSROOM INSTRUCTIONAL SYSTEMS IMPLEMENTATION 



COMPONENT 

/ 



DATA COLLECTION 



PERSONNEL 



I ~ Record Keeping 
Student Records 
CI ess Records 
Attendance 
Test 

II. Planning 
Resources 
Goals 

Activities 
Assignments 
Time Schedule 
Training Students 
Instructional 
techniques 
Large Groups 
Small Groups 

III. Material 
ELSP 

Curriculum 

IV. Monitoring 
Work Process 
Individual Curriculum 
Activities 

V. Assessment 
Student Needs 
Curriculum 
Grading System 
Student Self- 
Assessment 

Recycling 



Instructor's Log 



Instructor's Log 



Student 

Evaluation Report 



Criterion Reference 
Test 

Student Evaluation 
Report 

Instructor's Log 



CGP (Pre, Post) 
CRT 

Faculty Survey 
Student Evaluation 

Report ~ 
Instructor's Log 



Student/Instructor 



instructor 



Student/Teacher 



Student 



Studen t/ Instructor 



Instructor 
Student/ Instructor 



Student 
Instructor 



/ 

/ 



CHART 2 
TRAINING 



(continued) 



COMPONENT 


DATA COLLECTION 


PERSONNEL 


1. In Service 




ISE Staff 


PME System Implementation 
Beta Collection and Dissemination 
curriculum (ELSP; Implementation 
Process: 
Record Keeping 
Report Writing 
Hater tals 
• Testing Program 


ELSP 

Instruments* 


Participating 
Institution's 
Decision Makers and 
Information Users 


II. Training (Cross Institutional) 




ISE Staff/ Instruments 


Summer Workshops 
based on Identified Needs 


ELSP 

Instruments* 


Participating Insti- 
tutions 1 Staff 



* Collaborative Planning 



Test Instruments-Students: 

The Comparative Guidance and Placement (CGP) Test Battery wMrh | S 
prepared by the Educational Testing Service, Is designed to assist two- 
VMr, vocational technical Institutes and four-year colleges with open 
door admission policies* «nd In making decisions about currlcular 
guidance placement,. 

Five of the six placement Sests of the battery are Included In the 
ELSP assessment program: Reading, Written English Expression, Applied 
Arithmetic Computation and Elementary Algebra. Administration of the 
total test battery requires 1 hou/ and 50 minutes exclusive of the time 
for self-scoring 

The posttest battery of the assessment program Includes the use of 
the same form of the Comparative Guidance and Placement Tests Battery, the 
Student Evaluation Questionnaire and the Counseling Questionnaire. The 
Student Evaluation Questionnaire examines students' perceptions of the 
€ISP. Questions are raised about the kinds of learning activities provided, 
the leadership style of teachers, whether the course was related to the 
students' background of experiences, whether the course Implemented students' 
motivational systems and- the students' endorsement of the program. The 
Counseling Questionnaire alms to determine the students' awareness and 
utilization of ELSP counseling services, as well as the effectiveness of such 
services. 

The criterion-referenced tm*t package Includes a mastery test and a 
series of unit tests for each of the disciplines as follows: 
• Communication Skills 

Reading: mastery, 2 unit tests 
Wrlt'ig: mastery, 4 unit tests 
Mathematics: iwstery tests, 8 unit tests 
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Biology: mastery tests, k unit tests 



These tests, which s*;nple the extent to which students have mastered 
or have not mastered a specified set of objectives, are the foundation of 
the assessment program. The mastery test, which is administered prior to 
and at the end of each semester, facilitates diagnosis of the students 1 
mastery of the objectives for the entire course. The unit tests, which are 
administered prior to and at the end of specified Instructional sequences 
throughout the semester, assist faculty in (1) assessing the students 1 
. performance on an on-going basis and mastery of objectives throughout the 
semester and (2) prescribing responsive Instructional «plans, Including 
the use of support services such as personal counseling and academic 
tutoring. 

„ Each test package (standardized and criterion-referenced) includes 
guidelines for the administration and scoring of tests as well as the use 
of such Information In planning educational opportunities. " 




Information Survey Form. Specific catergories of student characteristics 
investigated include demographics (i.e. race, age, financial aid, education 
and occupation of parents); self-concept, feelings about racial preferences 
regarding the delivery of educational services; and educational plans (Apoendix B) 
Personnel : 



The Faculty and Administrator Survey Form Is designed to provide back- 
ground information on all staff participants in the ELSP. The survey form 
investigates four categories of behavior regarding faculty and administrators: 
background of experience (e.g., training, previous work experience); current 
position, including academic rank and teaching load; orientation to the ELSP; 
and attitudes toward selected educational goals (Appendix B) 




, Background information on students Is provided through the Student 
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Management Reports: 

Currently, the Information system of the ELSP assessment program includes 
seven reports. (See Appendix B) 

The H-1 report summarizes the results of the Comparative Guidance and 
Placement tests and can be used at the program level and the course level. 
This report Is- used to randomly assign students to classes as well as to 
determine the performance level of students from a global perspective upon 
entry Into the program. At the course or Instructional- level, this report , 
Is used to summarize the standardized test scores of Individual classes. 

The M-2 report Is the Class Performance Analysis which summarizes the 
performance of students on the cr iter Ion-referenced tests by class and In- 
dividual student. This report Is used to diagnose student mastery of ob- 
jectives and to prescribe the appropriate learning opportunities. 

The M-3 report Is an Item Analysis which further facilitates the diag- 
nostic process and curriculum revision. This report facilitates Identification 
of items which need to be revised, deleted or retained as written. The 
report also provides assistance In Identifying difficulties of the class as 
a whole and analyzing student and class performance on each .Item. 

The M-4 report is a comparison of pre and post CRT scores and assists 
faculty in determining the growth of Individual students and the class at 
the end of each period of treatment based on the total number of correct 
answers. The report provides the pre-test and post-test scores for each 
student, and, further, the percentage gain. 

The M-5 reports on the frequency distribution of total correct 
responses and presents the distribution of total rorrect responses by 
number and percent of students. This report is useful in determining the 
axtent to which the class mastered a selected group of objectives for the 
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mastery tests and Interim tests as well as variability In scores and making 
comparisons between pre* test and post* test data. 

The M-6 report gives cumulative percentage gain In CRT performance, 
summaries the percentage gain achieved by individual students and class for 
all CRT's and the cumulative gain sustained over a per'od of time. This 

s 

report Is useful for assessing 1 the progress which students have made during 

the semester. 9 

* — — — _ 

The M-7 report facilitates the summarizing of cost data for the ELSP 

• • . - — _ 

per semester. 

M-l Is primarily for the use of program directors and data coordinators; 
M-2 through M-6, for faculty In support of the instructional management 
, process; and M-7, for the program directors and data coordinators. 

Three forms other than the management reports themselves are included 
in the assessment program. A referral form is provided for requesting 
counseling resources. This form prescribes the kinds of services required, 
including the Identification of objectives and related instructional materials . 
and techniques. The bottom of the form Is for summarizing the results 
of such assistance. Including the objectives mastered or not mastered, the 
materials used, and recommendations for further improvement. 

The remaining two forms are used for summarizing class attendance 
and attrition by month and the reason for withdrawal. All three of these 
forms are to be incorporated in the faculty log, a document used in the pilot 
study for documenting the results of the criterion-referenced tests. 

A pre-assessment questionnai re (See Appendix P) is utilized by ISE 
personnel to obtain information regarding students and participating in- 
stitutions prior to program implementation. 

Appendix C Illustrates a flow of information process for managing the 
project. 
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Institute for S trv lets to Education's Strategies for Implementation 



of Progtyam Coals 



Th€ Institute for Services to Education In cooperation with participating 
Institutions will develop a «odel for the Implementation of program goals. 

Following Is a list of the five major goals of the ELSP project and 
recommended actions to Insure optimal program Implementation. 

GOAL: To develop a planning management and evaTuafToiri?M^^ 

*hich is compatible with the institution's Planning, Management, ~ — — 
and Evaluation (PME) system. 

ACTIONS ; 

• Assessment of the institution's current PME 

• Dissemination of results to decision makers 

• Collaborative planning to assist institutions in the development 
or ref fnement of current PME 

• Oeveiop custom-tailored ELSP PME that is compatible with the 
institution's system 

• Training workshops focusing on the ELSP PME system 

• Periodic evaluation of the effectiveness of the PME and to make 
recommendations to upgrade and update the system. 

GOAL: To implement a research evaluation plan for ELSP that is compatible 
with the institution's planning, management, and evaluation (PME) 
system. 

, ACTION: 

• Assessment of the level of implementation of ELSP 

Dissemination of Results to decision-makers 

Collection of data (e.g. f implementation process, student data, 
supportive programs) 

• Analysts of data 

• Document, publish, and disseminate results 

Collaborative planning to assist in the refinement of the imple- 
O mentation process. 
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GOAL: To assist participating Institutions In the establishment and Imple- 
mentation of a viable data collection, processing, analysis and 
reporting system. 



ACTION: 



Assessment of the Institution's current approaches to data manage- 
ment Including assessment of resources and capabilities 

Collaborative planning with the Institutions' administration and 
support staff regarding approaches to data management and dissemi- 
nation of the results of assessment 

Assistance In Implementing « data management effort for the ELSP 
project that Is compatible with the Institution's caoabilltv * n H 
resources 

Assistance In establishing lines of communication and authority 
to maximize the data management effort 

Provide Institutions with requisite forms and Instruments for 
data collection processing, analysis, and reporting 

Conduct training workshops focusing on the data management effort 

Periodic assessment of Implementation process, make recommendations 
for updating and upgrading data management 

Monitor datj management effort on an ongoing basis 

Publish and disseminate Information to the Institution's decision- 
makers. 



GOAL: To assist In the development .of instructional systems (i.e., 
structlonal strategies, recordkeeping, classroom management) 



ACTIONS: 



Assessment of the level of curriculum (ELSP) implementation 

Dissemination of results to decision makers 

Collaborative planning with administrators and faculty 

Development /refinement of Instructional approaches, and 
classroom management practices 

Periodic documentation of classroom implementation and ori-going 
feedback of findings 



* Coordinate and implement training workshops 
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GOAL: To provide on-going In-service training and summer workshops focusing 
on individual and group needs of program participants. 



ACTIONS: 



Assessment of training needs 

Dissemination of results to decision makers 

Collaborative planning to develop a priority list of training 
needs. Development strategies to improve skills of program 
participants 

• • • 

Coordinate and Implement training 

Document, publish, and disseminate results and training efforts. 
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APPENDIX * 



QUESTIONNAIRE USE* BY CURRICULUM 
FACULTY DEVELOPMENT STAFF 



Rationale; This questionnaire attempts to get at: 

- Whether students are adequately prepared for col lege 
work, at the latest, by the end of their first three 
semesters 

- Instruments used to assess this preparation . .. 

- Kind of data available that will demonstrate the 
degree of preparation 

- Creation of an awareness of how to meet the needs of 
students poorly serviced by the sfco^rysch??! 
system, 1n terms of programs, methods, materials, 
monitoring, etc. . 

1. What percentage of _ ^_ 's full time freshman have ninth 

grade level or below skills in: ^ Tests used 



a. English usage 

b. Reading 

c. Quantitative reasoning 



This supposes measurement by use of standardized achievement tests • 
or diagnostic measures. 

- Do.. have a policy pertaining to minimal level 

Sffll s required \n the above categories for going beyond the second year 
of colle|e? If yes! then pleasS describe. (Use supplementary sheet) 

<» to »d«rfnlstrators and supervisory personnel responsible for general 

If yes. please explain. (Use supplementary sheet) 

4 Are student services 1n place to supplement the above a. personal 
and vocational counseling, tutorial services, training leading to 
improved study skills, etc. 
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APtPfOIX A (eont.) 
togt 2 



Quest 




Ire used by Curriculum 



racui 



Development Con't 



What kind of data are collected or methods used to: 1} ensure the 
correct selection In the three skill areas Identified in item 1 and 
2) facilitate monitoring and evaluation of performance In the areas 
described? 

Can you estimate additional cost, if any, that these programs entail 
over the general curriculum?. If so, what is the estimated dollars 
or percent of additional cost per student? (Use supplementary sheet) 

Does the above impact on the numbers and percentages of students: 
1) graduating; 2) going on to graduatejand professional schools? 



This questionnaire is designed to help ISE Curriculum and Faculty 
Development staff assess: 

1- tne degree tb which students' academic needs are recognized 

2- the degree to which programs are in place to meet these needs. 
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Questionnaire used by Curriculum 
Faculty Development Stiff Con't 



SUPPLEMENTARY SHEET 



< APPENDIX B 

P 

Management Reports are basically tiMirltt. comparisons and 

» 

statistical analysas of studant performance on tha CCPP and CRT testing 
consonants. These reports which are generated periodically for both the 
Pretest and Post-test stages are key items in the program evaluation stage. 

The Management Reports are listed below and samples are provided 
on the following pages. 

Ml - Results of Comparative Guidance and Placement Test (CGPT); 
. M2 - Class Performance Analysis (on the CRT); 
M3 - Item Analysis (on the CRT) ; 
- Mil - Comparison of Pre and Post-test CRT Scores; 

i 

MS - Frequency Distribution of Total Correct Responses? and 
M6 - Cumulative Percentage Gain In CRT Performance. 
Other Source Documents 

^Other Source Documents provide information on students, faculty 
and institutions. These documents also are key elements of the program. 

The source documents which are to be generated are listed below, and 
samples are provided on the following pages. 

Student Information Survey Form 
Faculty and Administrator Survey Form 
I nst i tut lona 1 I nventory 
Faculty Log 
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HI 



h Entr/Uvtl Stills Progrw 
Prt-Ttst: tostUtt: 



•f tettlWitlM: 



_p*U UoNImUt: 



Cwrtt Title: 



Sect I m; 



ImlntcUr: 



SIO 1.0. 



NAME 



fYL 
CO! 



REAOMB 

sir 
sea 



KtC 



WlffU ENGLISH W 



TIL 
CM 



SIP 

sa 



Am ICO AMI TIMET IC 



IH 
COR 



SIO 
SCR 



KRC 



III. 
COR. 



CQNPUIAIION 

SIO 
SCR 



flRC 



CllMEMIARV AL6CRRA 



III 
COR 



SIO 
SCR 



rate 



CLASS SUtMRV: 



MEAN 
S.O. 
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CRI 



filial 



nz 

V *LASS PERFORMANCE CRT ANALYSIS 
\ Ulry Iwil SIIIU fnflTM 
*«H§»t: Ntttttt:. 



CUtt Smury: NtM I CM: 
S.R: 



HeM COR: 
S.t.: 



Hills 



Jte. •? StwiMtt:. 



\ 



. J». of Item:. 



_SmUm: 



jMkJtct: 



iMtincUir: 



SIN I.N. 



-it 



JL 



Ji, 



in 



in s III 



(COR ICOR COR COR N-A 



\ 



\ 



\ 
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M3 

ITEM ANALYSIS CRT 

Entry Ltvtl Skills ProyrM 
frt-lttt: Poslttsli 

CW ■».: ^-J»1H_ P*f fc*U p. of SUmImUs Jto. of list IUks: 

fours* I lilt; Section: . Sufcjoct: Instructor: 





Ill CLASS 


UfffA TIIIAO 


LOUEA IHIAD 




A 1 C • E M-A SMC SCOA OISC 


It «i A 1 C M M flMC ICOH DISC 


• 

Mm A rr. C 0 E N-A IMC ICOH DISC 


1 








2 


> 






3 


- 






4 








S 




r 




• 








1 


• 




* 


■ 








• 
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* 




II 
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Ml 

Caparison of Pre- and Post-CKT Scores 
Entry Level Skills Progran 
Mastery Test: Intertai Test: 

CUT y„| t: Pre -Test Date: Mo. of Students: Posttest Date: Mo. of Students:. 

Cmrse Title: ■ Section Mo.: Subject: Instructor: _ 



Student 1.0. : Student Naae Pre-Test Posttest Main 



/ 



Class SuwHiary: MCOR MCOR Mean Gain 
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, <Sf S.D. S.O. 



H5 

FREQUENCY DISTRIBUTION OF TOTAL CORRECT RESPONSES 

k 

Entry Level Skills Prograa v 
Mastery: Inter!*: 



m unit: Date of Test:_ Ho. of Students: Ho. of Ue-s:. 



c ub w t . _Sect1on: Jnstructor:. 

Course Title: , buojeci. 



TTL SCO Ho. of STU 



X of STU 



STATISTICS: 

Mean:_ 

S.O.: 



Range:. 
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REFERRAL FORM 
Entry Ltvtl Skills Program 



Cite of Rtftrral : 



1.0. Number: 



It T1tlt: 



Subject: 



|Th1s student needs help 1n the following: 

Prescription 



instructor :^ 
Section : 



jjectlve.f 1 



Objective # Z a 



Objective #3 



.Tutor: 



J. ..Student mastered Objective(s) #_ 
^. Used prescribed materials 



Results of Referral Service 
> Tutoring Period: 



Used other materials 



LL-_ Student, d_1d not master Object ive/(s)- # 
►J __ftacommendat1 on i ■ 



Used both 



3. Student needs help with entry behaviors relating to Objectives * m 
.Counsel or: 



Date: / / 



9 
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INSTITUTE FOR SERVICES TO EDUCATION 
STUOENT INFORMATION SURVEY FORM 

ENTRY LEVEL SKILLS PROGRAM 

1 

... 

Please Print: Your flame ' 

Social Security Number ( ( 

Current Address 

Number and Street 

' v 

~Hty State Up tod* ' 

Permanent Address . 

Number and Street' " ~ 

City ; State : ! ZTo Code 

Date of Birth Place of Birth 

MontnOay Year 



1 .ERJC 



52 



STUDENT INFORMATION SURVEY FORM 

DIRECTIONS: The Information requested 1n this survey form will help us to 
learn more about your background, educational plans, and feelings 1n regare to 
your potential for academic success. Please respond to each question care- 
fully- 6 

Your response will be read by an optical scan reader. Your careful 
observance of these few rules will be very Important: 

(1) Make heavy marks that fill the circles. 

(2) Erase cleanly any answer you wish to change. 

(3) Write only 1n designated areas. 

• • • 

(4) Answer all applicable questions. 



1. Nam* of.hloh school last attended: 

Anacostla 

Wilson * 

Splngarn 

Ounbar 

Cardozo - . 

Roosevelt 

Coolidge . 

Chamberlain Vocational 

Bell Vocational •_ 

Other (Specify) 

2. The geographic location of your last high school was in: \.. 



S.E 



J 






N 


.w 


9 


N 






S 


:a 





° Suburban 

Other (specify) 
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1 high school wu.* (Mw* ont) 

All or predominantly white ...... . 

All or predominantly black 

4. Old you graduate from high school: Yes 

No 

5. If k yes, give the year you graduated: 

1979 

1978 

1977 . 

1976 

1975 

1974 

Other (specify) -. / '. . 

6. Race: 

White . 

Black . 

Hispanic . 

Asian . . 

Native American : . 

Other (specify) 

7. Number of dependent children: 

One Child : : 

Two. 

Three 

Four 

Five or more ♦ 

None 
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Currently wp>oyed full time 

Currently employed part time '.' r 

. Currently unemployed . . * . 

9. Employment status of spouse: 

to 4 * 

enployed ; . \ . - 

Unemployed 

10 - Family income level: 

V 

Less Than $4,000 a Year 

* . . * '.* 

^ 54,000 - $5,999 s. .....<"'..'. 

\ ** . ^^^^ 

56,000 - 57^999 «/. ......... . 

58,000 - $9,999 ...... f. ........ . 

510,000-512,999 \ J • 

513,000*514,999 . . r -. . 

515,0d0-$16,999' * 

517,000-518,999 . 

a 519,, 000 and Over - 

11. What Is the dollar amount of the scholarships, qrants. loans and/cr college 
work-study which you presently receive? Write dollar amount in the soace 
below *or each kind of aid you are receivinq this year. Write none 1f 
you are not receivinq this kind of aid. » 

Type of Aid Amount 

Basic Educational Opportunity Grant 

Supplemental Educational Opportunity Grant 

State S^ho^rship/Grant 

College Grant (other than above) 

Federal Guaranteed Student Loan 

National Direct Student Loan 

Other College Loan 

Other Loan (specify) 

CollegV^ork Study Grant 
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♦ Typf of Aid ^Continued) Amount 



6. I. Benefits . 
Social Security 
None 



Other (specify) / 

State Aid (specify) 

12. What 1s the total amount of aid you are 

presently receiving from all sources: 



13. Do you receive any other supplemental 
Income from the following sources: 

Parents 

Other relatives 

Employment 

Loan 

Spouse . . 

Other (specify) 

14. Are you financially independent of your parents? 



Yes 



15. How many persons are currently dependent on 
your parents for support (include yourself 
and your parents, 1f applicable)? 

1 



«- 



2 
3 
4 
5 



6 or more 

16. How many of these dependents other than yourself 
are attending college? 

None 2 4 or mere 

1 3 
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17. What w, 



22i!?L? , f5 l8 S i,t edutat1onal 'j^l that each. of your parents or guardian(s) 
™ P I If vou are n ot sure, please give your best cuess . In the • 

1 Father" martc one space to show the hlghett educational 'eve; 

"Mrt^h*** h D i! Ur fa i her * J* the sa D e for y°" r mother In the column headed 
Mother. P|*ase answer this Question even If vour oarents or quardian(s) 
are deceased. \ 

(Mark one number 1n each column} 



Father 
or Male 
Guardian 



Some grade school or less\ 

Finished grade school. .. \ 

Did not complete high schooj (\^th grade) 

Some business, vocational, technical or trade school . 

Finished business, vocational, technical or trade school. 

Some college (including two-year degree^ 

Finished college (four-or five-year degree^ 

Attended graduate or professional school (fokexample, 
law cr medical school), but did not attain \ graduate 
cr professional degree A. . . . 

Obtained a graduate or professional degree (for example. 
M.A., Ph.D., or M.D.) >! . [ 

I don' t knew 

X 



Mother 
or Female 
Guardian 



\ 



\ 



\ 



\ 



\ 
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From /Che list^below, select the type of work whtfh comes closest to the 
job/your father o ^uaTTy has and mark the number \n the column headed 
"Father 11 . Then do theSwne^for the job your mother usually has and 
mark the number 1n the columft\headed "Mother" If ofcf of your parents 
or/ guardians is currently unemployed, retired, or deceased, pick the 
tj/pe of work this parent usually )did. 



(Mark one number in each col urn 



Father or 
Male 

Guardi an 



Mother or 

Female 

Guardian 



Laborer or Service Worker : s 
fara worker, bus driver, taxi 
worker, .construction worker, 
waitress* gas station attend 
cook J, qiaid, custodian, guard 
man, seamstress, beautician /or barber, 
practical nurse . • 



ch as factory or 
driver, mine 
aiter or 
t, gardener, 
fireman, police- 



Craftsman or Foreman : suctAas carpenter, 
mechanic, plumber, electrician, baker, 
mason, tile setter, paint/r, television 
repairman, machinist 



Office or Sales : such/as store clerk, 
bank teller, bookkeeper, mailman, rail clerk, 
office worker, secretary, telephone operator, 
real estate or insurance agent 



Manager or Owner : Such as farm owner, business 
owner, store or office manager, banker, govern- 
ment official , administrator 



Professional or Technical : such as teacher, 
doctor, engineer, lawyer, social worker, 
accountant, musician, dentist, registered 
nurse, librarian, artist, actor, writer, • 
engineering technician, science technician 
and health technician 

Homemakar or Housewife full -time 

I don 1 1 know 
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19. Is this coll tee your: 
(Mark one) 

First choice 0 
Second choice 0 
third choice 0 

Less than third choice 0 

\ - 

20. How many colleqes other than this one did you apply to? 
No other o 

1 0 

2 0 

3 0 

4 0 

5 0 

6 or more 0 

21. How many acceptances did you received 
(Mark onu) 

None C 3 0 

10 4 0 

2 0 5 or mors 0 

22. Cjd you take part in any of the following specia: high school programs .'out- 
side of regular school work) designed to help prepare you for or get you intc 
college or other types of scnools? If the exact name of the oro5r*n»<s) 

?2 t0 SJ*S art /if 1s not l1st ? d belo w " ? lsase w "*s the namats; in tHa sp acss 
after "Other (Please specify)". ' 

(f'ark one circle on each line) 



Sear: 



Project Opportunity 

Educational Opportunity Program (SO?) 
Project Access 



No 


Yes 


0 . 


0 


0 . 


0 


0 . 


n 

j 


0 . 


0 


n 


n 


n 


0 


0 


o 


0 .' 


c 


0 . 


0 



) 



Other (Please specify 
Other (Please specify" 



receive any of the following? 







0 . 


0 


0 . 


0 


0 . 


0 


0 . 


0 


0 . 


0 


you 





(Mark one circle on each Hne) 



No 



Yes 



Special courses designed to help ycu do better in high school 

courses or to prepare you for college courses o 

Tutoring 0 

Help with getting Into college or other types of. schools (such 
as Information about schools, help with applying, visits to 

campuses, etc.) 0 

Special experiences (such as field trips to museums, concerts or 

P^ays) 0 

Financial assistance for education beyond high school Q 

Training 1n a vocation 0 

Academic counseling 0 

Personal counseling \ q 

, Other (Please specify j . . . . \ q 

Other (Please specif y ) o 

24. How well do you feel that your high school Drepared you in the following 
areas: 

- (Park one 1n each row) 



0 
0 



0 
0 
0 
0 
0 
0 
0 



Mathematical skills 
Reading and composition 
Foreign language 
Science 

History, Social Science 
Vocational skills 
Music and artistic skills 
Study habits 



Very Well 
0 
0 
C 
0 
0 
0 
0 
0 



Fairly Well 
0 
0 
0 
0 
0 
0 

c 

0 



Poor ! 
0 
0 

• 0 
0 
0 
0 

c 
n 



\ 
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25. 



26. 



Hew do ypu rate the last high school that you attended with respect to 
following services? If a service was not available or 1f you don't knt 
1f 1t was available check the appropriate column. If service was avalll 
mark rating which best applies. 



ible 



(Mark only one circle on each line) 



Service 
Not 
Available 



Don't 
Know !f 
Service was 
Available 



SERVICE IS AVAILABLE A'JD IS: 



Poor Fair 



Excel - 
Good lent 



Can' 
Rate 



Kelp for students who are 
having trouble with sub- 
jects Tike math or read- 
ing 0 0 ' 0* 0 0 0 

Help 1n finding a job 0 0 0 0 0 0 

Counseling for personal 

. problems 0 0 0 0 0 0 

Career counseling 0 o 0 0 0 C 

Counseling on choice, 

admission, and financial 

aid for colleges or other 

types of schools 0 C 0 0 0 0 

How would you rate the last high school that you attended in each of the 
following areas? 

(Mark one circle on each line) 



0 
0 

0 
0 



Poor Fair 

Types of programs offered (academic, 

vocational , etc. ) 0 o 

Range of courses offered 0 0 

Range of extracurricular activities . 0 0 

Quality of teaching 0 0 

Teachers' interest 1n students .... 0 0 

Ccunselors' interest in students ... 0 0 

27. When did you decide that you ware going to colleae? 
(Mark one) 

Always knew C 

In elementary school 0 

In 7th, 8th or 9th grade 0 

In 10th or 11th grade 0 

In senior year of high school 0 

After high school graduation 0 

After involvement in pre-college program . . . . 0 

Several years after high school , Q 

None cf these 0 



Good 



0 
0 
0 
0 
0 
0 




0 
0 
0 
0 
0 
0 



Uncer- 
tain 



0 
0 
0 
0 
0 
0 
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\ 

\ 



2?. Do jfou have any concerns about your ability to finance your college 
. education? 
\ (Mark one) 

None (I am confident that I will have sufficient funds) .... 0 

Some concern (but I will probably have enough funds) 0 

Major concern (not sure I will have the funds to complete 
college 0 
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FACULTY ANO ADMINISTRATOR SURVEY FORM 
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FACULTY AND ADMINISTRATOR SURVEY FORM* 

DIRECTIONS: Tht Faculty and Administrator Survty Form 1s designed to provide 
background Information on all staff participating 1n the 1979-1980 pilot study 
of the Entry bevel Skills Program. No attempt will be made to Identify Individ- 
ual responses and Individual data will not be reported in any way. 



Your responses will be read by an optical scan reader, 
observance of these few rules will be appreciated. 

(1) Make heavy marks that fill the circles. 

(2) Erase cleanly any answer you wish to change. 

(3) Write only 1n designated areas. 

(4) Answer all applicable questions. 

SECTION 1. BACKGROUND INFORMATION 



Your careful 



1. Your age 1s: 
(Mark one) 

Under 20 0 

20 - 29 0 

30 - 39 0 



40 - 49 
50 - 59 
60 or over 



0 
0 
0 



2. Ave you (Mark all that apply) 

B1ack/Negro/Afro-Amer1can 0 

Amtrlcan Indian 0 

As1an-Amer1can/0r1ental 0 

Mexican American Chlcano 0 

Puerto R1can American 0 

White/Caucasian 0 

Other 0 



3. Do you have a: 

(Mark all that apply) 

BS/BA degree 0 

MS/MA degree 0 

Ph.D./Ed.D degree 0 

Other degree 0 

4. How long have you been work- 
ing at this Institution? 
(Mark one) 

First Year 0 

2-3 years 0 

'4-5 years 0 

5-6 years 0 

More than 6 years 0 

Not applicable 0 



* The Items 1n this instrument were drawn from the 1978-79 ISE Faculty and 
Administrator Survey Form and the Institutional Goals Inventory of the Educa- 
tional Testing Service. 
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Your employment status 1s: 

Full-time 9-month a*ppt. 0 

Part-time 12 month appt. 0 
Othtr 0 

Your academic rank 1s: 

Profossor 0 

Associate Profossor 0 

Assistant Profossor 0 

Instructor 0 

Nona of thoss o 

Your position 1s: 
(Mark all that apply) 

Faculty Person o 
Acidonlc Dean 0 

V1ca-Pras1dent for Academic 0 
Affairs 

Dopartment Chairperson 0 

Counselor 0 
None of these 0 

Your teaching load 1s: 
{Mark one) 

Less than 9 hours 0 

9 credit hours 0 

12 credit hours 0 

15 credit hours 0 

More than 15 0 

None of these 0 
Not Applicable 0 
Your tenure status 1s: 

a 

Tenured 0 
Non-tenured C 



10. Your najor field 1s: 
(Mark all that apply) 

Arts and Humanities 
Biological Sciences 
Business 
Education 
Engineering 
Physical Science ' 
Health Science 
Social Science 
Computer Science 
Communications 
Mathematics 
Other 

11. Your responsibilities other than 
teaching Include: 

(Mark all that apolv) 



Student advisement 0 
Coordination of student activities 0 

Committee chairoerson 0 

Comlttee member 0 

Administrator 0 

Consulting 0 

Public Service Activities 0 

Other o 
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BS/BA graduate. 
0 0 



0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 



0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 



12. Tour previous work txptrltnct 
Includes: 

(Mark all that apply) 

Elementary taachlng 0 

Socondary taachlng o 

Community College taachlng o 

Collaga/Un1van1ty taachlng 0 

Administration (Acadanlc) 0 

Non-Academic Administration 0 

Educational Research 0 
Nona of these 0 

13. You have had substantial on-hands 
experience 1n: 

(Mark all that apply) 

Curriculum design 0 

Writing behavioral objective: 0 

Constructing criterion- 
referenced tests 0 

Competency-based education 0 

Mastery learning 0 

Developmental education 0 

Remedial education 0 

Conducting educational 
evaluations 0 

Information system 

development 0 

Conducting educational 

research 0 

None of these 0 



14. You teach only: 

(Mark all that apply) 

Freshman courses 0 

Freshman and sophomore 

courses 0 

Upperclass courses 0 

Graduate courses 0 

All of these 0 

None of these 0 

Not applicable 0 



15. How many years have you 
worked with the ISE Currlcu 
lum and Faculty Development 
Program: 
(Mark one) 



Less than a year 
1 - 2 years 
3-5 years 
More than 5 
Not Aopllcable 



0 
0 
0 
0 

0 
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SECTION 2. ENTRY UVa SKILLS PROGRAM 



16. Is your rolt 1n tht ELSP that of: 
(Ntrk all that apply) 

Teaching faculty 0 

Couratlor 0 

Olrtctor of Program 0 

Oata coordinator 0 

Task forca member 0 

Nona of thasa 0 

17. What 1s your understanding of 
your rola 1n tht &SP? 

(Mark ont) 

Claar 0 

Somewhat claar 0 

Vary dear p 

Confusad 0 



18. Your Involvement 1n the pilot 
study 1s: 

Voluntary 0 

Mindatory 0 

Supportive 0 

19. In relation to other types of 
developmental education programs, 
the amount of work that you are 
required to do 1s: 

(Mark one) 



20. 



Moderate 

Manageable 

Excessive 



0 
0 
0 



Old you have an opportunity to 
participate 1n ELSP orientation 
and planning meetings: 



Yes 



No 



21. 



If yes, did you participate 1n: 
(Mark all that apply) 

Pre-Summar Conference planning 
by your Institution 0 

Prt-Summer Conference planning 
by ISE 0 



Conference 



Post-Summer Conference planning 
at 011 lard University 0 

Post-Suwner Conference orienta- 
tion and planning at your 
Institution 0 



You have been oriented to ELSP's: 
(Mark all that apply) 
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No 


Goals 


0 


0 


Basic Assumptions 


0 


0 


Instructional 
Materials 


0 


0 


Testing Procedures 


0 


0 


Recycling Proced- 
ures 


0 


0 


Focus on cognitive 
skills 


0 


0 


Accountability 
measures 


0 


0 


Interaction with 
other Institutional 
programs 


0 


0 


Standardized test 
competency 


0 


0 



22. Op you agree with ElSP's: 
(Mirk ell that asply) 

Yg No 

Soils 0 0 

Focus on Cognitive 

Skills 0 0 

Focus on standardized 

tost competency 0 0 

Tasting requirements 0 0 

Documentation require- 
ments 0 o 

Design Process. 0 o v 

SECTION 3. EDUCATIONAL GOALS 

Directions: Indicate th*t extent that you agree with the following statements' 
by responding Strongly A$ree (SA), Agree (A), Olsagree (D), Strongly Olsagree (SD) 
or uncertain (U). 

The ELSP should give special attention to these educatloal goals: 

. 23. To Increase the desire and ability of 0 0 0 0 0 

students to undertake «elf-d1rected study. SA A 0 SO U 

24. To help students Identify their own 

personal goals and develop means of 0 0 0 0 0 

achieving then SA A 0 SD U 

25. To help students prepare for advanced 

academic work (e.g., graeuate and 0 0 0 0 0 

professional school). SA A D SD U 

26. To encourage students to make concern 

about, the welfare of all mankind a 0 0 0 CO 

central part of their lives „ SA AD SD U 

27. ' To provide opportunities for students to 

prepare for specific occupational 

careers (e.g., engineering, health pro- 0 .0 0^ 0 0 

fesslons, business administration) SA A 0 SO U 

28. To encourage students to become con- 
scious of the Important moral Issues of 0 0 0 0 0 
our times. SA A 0 SD U 

29. To help students develop a sense of self- 
worth, self-confidence, and a capacity 0 0 0 0 0 

r D ? r to have an Impact on events. SA A 0 SD U 

68 



30. To teach students Methods of scholarly 
Inquiry, scientific research, and/or 
problem dtflnltlon and solution. 

31. To Instill In students a Hfalong counlt- 
Mnt to 1 taming. 

32. To Insurt that students who graduate have 
achieved some level of reading, writing 
and mathematics competency. 

33. To ,55 111tttf Involvement of students In 
neighborhood and community-service 
activities. 

34. To provide educational experiences relevant to 
the evolving Interest of woman In America. 

35. To help people from disadvantaged communi- 
ties acquire knowledge and skills they can 
use In Improving conditions In their 
communities. 

36. To help students bring about change In 
American society. 

37. To Insure that students acquire basic knowl- 
edge In the humanities, social sciences and 
natural sciences. 

38. To help students In deciding upon a 

vocational career. 

39. To help students be open, honest and trust- 
ing In their relationships with others. 

40. To Increase students sensitivity to and 
appreciation of various forms of art 
and artistic expression. 

41. To contribute, through research, to the 
general advancement of knowledge. 

42. To develop students ability to synthe- 
size knowledge from a variety of sources. 



0 


0 


0 


0 


0 


5A 


A 


D 


SO 


u 


0 


0 


0 


0 


0 


SA 


A 


0 


so 


u 


0 


0 


0 


0 




SA 


A 


0 


so 


u 


0 

SA 


0 
A 


. o 

0 


0 

so 


0 

u 


to 0 


0 


0 


0 


o 


SA 


A 


0 


so 


u 


0 


0 


0 


o 


n 

V 


SA 


A 


0 


so 


u 


0 


0 


0 


0 


0 




A 

A 


0 


SD 


u 


0 


0 


0 


0 


0 


SA 


A 


0 


so 


u 


0 


0 


o 


n 


n 
u 


SA 


A 


0 


so 


U 


0 


0 


0 


0 


0 


CA 

5A 


A 


0 


so 


u 


0 


0 


0 


0 


0 




A 

n 


n 
U 


en 


u 


0 


0 


0 


0 


0 


SA 


A 


D 


SD 


u 


0 


0 


0 


0 


0 


SA 


A 


D 


Sd 


u 
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INSTITUTE FOR SERVICES TO EDUCATION 
Entry Level Skills Program: Faculty Log 



Name: 



1.0. Number: 
Subject: 



Name of Institution: 



Location of Institution: 
Academic Year: 



Title: 



Course Title: 
Section: 



— m*) 

Semester: First 



(State) 



Second 



o 
RIC 
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II. CLASS PERFORMANCE SUMMARY 
Standardized Tests: 

Reading 

Written English Expression 
Applied Arithmetic 
Computation 
Elementary Algebra 



Pre-Test ( « / ) Posttest ( »/ ) 

Mean S.D. Mean S.o/ 



Criterion-Referenced Tests (Mastery) 
Units to 

MINC 
MCOR 
MINC 
MCOR 

COMMENT: 





%INC 
S.D. 
XINC 

S.D. 



XGaln 



53- 



1 



Course and Number 



SUMMARY OF INTERIM (UNIT) TEST RESULTS 
Entry Level Skills Program 
First Semester: Second Semester: 



Instructor's Name 



Type of Administration 

Date 
Pre-Test: 
NINC 
XINC 
NCOR 
S.O. 
Post-Tes*t: 
NINC 
XINC 
NCOR 
S.O. 
ZGAIN: 
COMMENT: 



Interim Tests 

<-/_/_) (_y_y_) (_/_!/_) u*j-) u*j_j l/L/j u_j_j u_j_) 
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CLASS PRESCRIPTION/CURRICULUM RECOMMENDATIONS 
Entry Level Skills Program 
Unit Objectives 



Course and Number 
No. 

Decrease Hen Mast- 
ery percentage 

Increase class Mast- 
ery percentage 

Increase class Mast- 
ery and decrease Item 
Mastery percentage 



Unit Objectives 



Instructor's Name 



Delete these objectives 

RecoMMended Changes In 
Instructional Materials 

Broaden scope of con- 
tent 

Enhance readability 
level of content 

Lower readability level 
of Materials 

Incorporate no re exper- 
iences relevant to 
students 

Enhance use of educa- 
tional technology 

er|c 

















































1 

1 
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i 
I 



Course and Nunber 



MONTH 



Total Number of 
. Students 



SUMMARY OF CLASS ATTENDANCE 
Entry Level Skills Program 
First Semester: Second Semester: 



Total Number of 
Class Sessions 



Percent of Class 
In Attendance 



Instructor's Name 



Number of Drop-outs 



August 

September 

October 

November 

December 

January 

February 

March 

April 

May 

COMMENT: 
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Course and Hunker 



SUMMARY OF STUDENT ATTRITION 
Entry Level Skills Program 
First Semester: Second Semester: 



Instructor's Name 



Student I.D. 



Name 



Date Dropped Out 



Reason Dropped Out 



f 



-82- 



-ERXC 



OBSERVATIONS 



In the process of any given classroom experience^' a number of events occur which were not 
anticipated or which deserve special note because they were Inconsistent with one's expectations 
(high or low) or they document or Invalidate hypotheses which have been formulated. Examples of 
such observations Include:' 

too much or too little time specified for accomplishment of a given objective; 

* student comm en ts related to the effectiveness of a particular activity, Instruc- 
tional mode or new Idea; 

' high or low Interaction level for a particular learning activity} 

' a particular method that students used In group sessions to facilitate learning, 
understanding; and 

the developmental stages In student attitude formation (attitude toward learning 
generally, test taking, other students, other courses). 

Additionally, faculty may wish to use the log to record progress made each day. along with 
recommendations for Improved Instructional planning. 
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Date 



Objectives 



Observations 



0 



APPENDIX C 

The flowcharts presented on the following pages suggest a flow of 
Information process for managing the project. The charts which are self 
explanatory trace the movement of most management reports and other 
source documents from their point of origin through the various reponsible 
officials. Students faculty, counselors, data coordinators, program 
directors and ISE consultants all play a role In either generating, re- 
viewing or analyzing project documents. The charts also ind*cate the 
number of copies of each document that are to be made available and 
Identifies the office responsible for retaining each document. 

It is essential that the role and responsibility of cached? vidua 1 
involved In the process be clearly established as early as possible to 
Insure adequate accountability and the efficient flow of information. 
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PLOW 



students 



FACULTY 



C •UNSELOft 



DATA 
COORDINATOR 



PROGRAM 
DIRECTOR 



INSTITUTE FOR 
SERVICES TO 
EDUCATION 



CC#T» 
Answer and 
Bmm ar y Score 
Sheets 



CGPTB 
Answer and 
SuMMiry Score 
Sheets 



CGPTB 
Answer end 
SuMury Score 
Sheets 



CGPTB 
Answer end 
Gunnery Score 
Sheets 



M-l 






■ 




Report 




* 


Report 






1 






1 
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FACULTY 



COUNSELOR 



DATA 
COORDINATOR 
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Form 
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Administrator 
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1 




r 

Faculty 

Administrator 

Porn 

1 
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PROGRAM 
DIRECTOR 
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Faculty 

Administrator 
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Student 




Survey 




Information 
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1 



Student 
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Student 
Survey 
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EDUCATION 
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CRT 










Anwar 




Sheets 




} 


Sheets 






1 






1 



CRT 




Answer 
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1 



3 



CRT 




Answer 
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APPENDIX 0 



The Information management system of the ELSP readily lends Itself to 
manual as well as automated data processing techniques. To facilitate this 
a data elements directory for the project has been designed. The directory 
has the following purposes: 

• Provides a central source of Information pertinent to the 
program; 

* provides a uniform approach to data collection and dissemination; 
0 provides a system for classifying and summarizing data; 

• provides, a mechanism for Identifying all test Instruments and 
other documents utilized In the programmatic and administrative 
aspects of the project; 

0 provides a system for Identifying and defining all data elements 
(information Items) found In the project documents; 

* provides a basic structure for the development of a data base 
(computerized system) for the project. 

The data directory along with a users guide can be made available to 
participating Institutions. 

A coding structure that can be utilized In the automated data pro- 
cessing effort appears on the fol lowing pages. The coding struct ire identif 

x ■ 

tadh document, end data element within these, with a unique numeric or alpha 
numeric codfe. The numbering system is quite flexible and allows for the 
addition or deletion of particular documents or data elements as necessary 
without altering the essential character of the coding sequence. 
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